An Interpretive Model of Hand-Eye Coordination

Overview Normal cognitive function requires coordination between perception and action. However, both action and perception pose enormous challenges to our scientific understanding, and, as a result they are often studied separately, with experimental paradigms carefully crafted to tease them apart (e.g., gaze fixation). We present an interpretive model that addresses both faculties within a single framework. A simplified task of hand-eye coordination is modeled as a stochastic optimal control problem, and the solution is a coordinated motion of eye and hands, derived from first principles. Our numerical results reveal a rich behavioral repertoire, with both smooth pursuit and saccades emerging as components of the optimal solution to the eye-hand control task. The Model We present an abstract model of a reaching task – two free-moving points (“hands”) move on a twodimensional plane (the “scene”), which includes four obstacles and a target (depicted in figure 1). The hands start at both sides of the bottom of the scene, and at a final time they are penalized proportional to their distance from the target, located at the top-middle of the scene. Each hand moves through a pair of obstacles on its path to the goal. The linear dynamics of the hands are subject to constant process noise, and the agent is provided with noisy observations of the state of the scene. State estimation and feedback control of the hands are therefore necessary to complete the task. The positions of both obstacles and target are fixed, but the learning agent starts with an uncertain estimate of these positions, and so they have to be observed and estimated as well. The eye is involved in the observation process – it provides information that allows for better state estimation, thereby improving feedback control and so contributing to task performance. Motivated by this abstraction, we model the eye as a “point of gaze”, a free-moving point in the scene. The effect of this gaze is a local modulation of the observation noise: if an object (hand, obstacle or target) is close to the point of gaze, the noise perturbing the observation of the object’s position will be reduced (this reduction being a Gaussian function of distance from the gaze point, as illustrated in figure 2B). Therefore, directing the gaze at an object allows the learning agent to generate better estimation of its position, which yields more accurate feedback control of the hands and hence better overall performance. Solution Since the observation noise is state-dependent (and hence not Gaussian), estimation requires a nonlinear filter (we employ the Extended Kalman Filter); furthermore, since the cost is not quadratic (due to the obstacles), the Stochastic Optimal Control (SOC) problem cannot be solved using standard LinearQuadratic-Gaussian techniques. Instead, we solve the SOC using Minimax Differential Dynamic Programming (the complete mathematical description was published in [1]; see references therein). The optimal solution is a feedback policy for controlling the positions of both hands and eye through the scene that exhibits both saccades (fig. 3) and smooth pursuit (fig. 4). In order to make this high-dimensional optimization problem tractable, we shape the solution by gradually reducing the width of the eye’s “fovea” (the width of the Gaussian inside which observation noise is reduced) – at first, wider fovea allows the agent to detect and learn what areas of the scene are most relevant at every stage; eventually, a narrower fovea leads to more distinct eye motions.